Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Chinese description of image content based on fusion of image feature attention and adaptive attention
ZHAO Hong, KONG Dongyi
Journal of Computer Applications    2021, 41 (9): 2496-2503.   DOI: 10.11772/j.issn.1001-9081.2020111829
Abstract334)      PDF (1520KB)(399)       Save
Aiming at the problem that the existing Chinese description models of image content based on attention mechanism cannot focus on the key content without weakening or missing attention information, a Chinese description model of image content based on fusion of image feature attention and adaptive attention was proposed. An encode-decode structure was used in this model. Firstly, the image features were extracted in the encoder network, and the attention information of all feature regions of the image was extracted by the image feature attention. Then, the decoder network was used to decode the image features with attention weights to generate hidden information, so as to ensure that the attention information was not weakened or missed. Finally, the visual sentry module of self-adaptive attention was used to focus on the key content in the image features again, so that the main content of the image was able to be extracted more accurately. Several evaluation indices including BLEU, METEOR, ROUGEL and CIDEr were used to verify the models, the proposed model was compared with the image description models based on self-adaptive attention or image feature attention only, and the proposed model had the evaluation value of CIDEr improved by 10.1% and 7.8% respectively. Meanwhile, compared with the baseline model Neural Image Caption (NIC) and the Bottom-Up and Top-Down (BUTD) attention based image description model, the proposed model had the evaluation index value of CIDEr increased by 10.9% and 12.1% respectively. Experimental results show that the image understanding ability of the proposed model is effectively improved, and the score of each evaluation index of the model is better than those of the comparison models.
Reference | Related Articles | Metrics